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ABSTRACT 


Each of the domains of HSI is, of itself, a discipline with vast amounts of 
research, analytic techniques, educational programs, and methods for evaluating the 
effectiveness of the system with respect to the specific domain. Relatively recently, 
domains with a logical similarity have been the focus of interest for researchers studying 
the plausibility of creating evaluative tools which take into account the constraints of 
multiple domains. This interest has led to the creation of various tools with which 
acquisition professionals can more accurately determine the impact of design decisions 
on the system as a whole. However, no single tool has yet been created which takes 
into consideration the constraints of all the domains which HSI encompasses. The 
development of such a tool would give decision-makers the ability to quickly and 
accurately determine the system-wide trade-offs associated with changes in a single 
domain. 

In order for this to occur, an in-depth study of the current tools associated with 
each of the HSI domains must be conducted. The most accurate tools from each 
domain must be integrated with a single interface. However, this step will only be 
realized after a common language has been identified which can speak to the 
effectiveness of the system in each of the domains. Finally, the human interface with the 
tool must be intuitive, and designed with the end-user in mind. 

This study identified the various resources currently available for evaluating each 
of the HSI domains. These resources were compiled in a searchable database for use 
by the HSI professional in the planning of HSI evaluations. Following a description of 
how HSI relates to the Department of Defense acquisition process, the design effort to 
produce an overarching interface was presented. This interface would allow the 
acquisition professional to evaluate the trade-offs between all relevant domains and 
make well-informed decisions with respect to the overall effectiveness of the human in 
the system. Next, a plan for insertion of the process and software into the acquisition 
community, making the tool available to all acquisition professionals, was discussed. 
Finally, as with all research, the limitations of the present study were discussed, as well 
as recommendations for future research. 
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I. INTRODUCTION 


A. HUMAN SYSTEMS INTEGRATION 

In its broadest sense, Human Systems Integration (HSI) is the effort to 
make human considerations the top priority in systems design. This broad 
definition, however, does not escape the grasp of the more global study of 
systems engineering. A system is defined as “a group of interacting, interrelated, 
or interdependent elements or parts that function together as a whole to 
accomplish a goal” (Massachusetts DOE, 2001). The basis of systems 
engineering—systems thinking—requires that any product or service be viewed 
as a piece of a larger system in order to better understand its requirements and 
characteristics. Sage and Armstrong (2000) describe systems engineering as 
having three key components: organizational, technology-based, and humans. 
As some areas of responsibility—and the interactions between the three 
components—tend to involve more than a single component, visualizing systems 
engineering as a Venn diagram may prove useful (see Fig 1). 


SYSTEMS ENGINEERING 



Figure 1. Three key areas of Systems Engineering 

Each of the key components of systems engineering falls under the 
purview of existing disciplines. While systems engineering efforts incorporate all 
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three components, the products differ from that of other engineering disciplines. 
Systems engineering produces abstract systems, and relies on the other 
engineering disciplines to design, build and test the tangible products. These 
well-defined disciplines have explicit knowledge of their specific domains and 
thereby design products to exacting details. 

The organizational component of systems engineering may be viewed 
through the discipline of macroergonomics. Macroergonomics grounds itself in 
sociotechnical systems theory, with the chief concern being human-organization 
interface technology—specifically, the analysis, design, and evaluation of the 
work system. Dr. Hal W. Hendrick (2002)—considered the “father” of 
macroergonomics—describes the discipline’s goal as “[optimizing] the work 
system’s design in terms of its sociotechnical system characteristics” and 
ensuring that these characteristics are carried down throughout the design of the 
sub-system efforts. 

The traditional engineering disciplines are encompassed by the 
technology-based component of systems engineering. Practitioners in 
disciplines such as electrical, mechanical, and aerospace engineering apply their 
expertise in designing systems to meet detailed specifications. While these 
specifications ensure the proper functioning of the system with respect to each 
individual discipline, they do not necessarily ensure the system will function 
properly with the additional systems with which it may need to operate. This 
level of integration is left to systems engineers. 

The final component in systems engineering—humans—falls under the 
purview of HSI. Where systems engineering focuses on the defining, developing 
and deploying of large scale systems, HSI champions the human element within 
systems design. It has been argued that focusing on the human element of a 
system is the most likely method for increasing system performance and 
reducing system life-cycle costs (Booher, 2002). The exact boundaries and 
areas of responsibility of HSI are not yet completely defined. However, the 
defense acquisition structure rests comfortably upon the foundation of HSI. 
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B. BRIEF HISTORY OF HUMAN SYSTEMS INTEGRATION 

Just as the trade space for systems engineering incorporates multiple 
fields of study, the present trade space for Human Systems Integration 
coordinates the efforts of several distinct disciplines or domains. These domains 
from which HSI has emerged have existed for many years. Though known by 
different names, and having evolved over time, the fields incorporated in HSI 
have always tried to answer the same question—how can the human be more 
effective in the system ultimately resulting in enhanced overall system 
performance. In fact, the predecessors of HSI in the U.S. military date back to 
the early years of the Army Air Corps, and the efforts to improve the performance 
of systems (aircraft) specifically at the points where humans interfaced with the 
machine (Chapanis, 1959). This initial effort resulted in the birth of human 
factors. More detailed histories of human factors can be found by Meister (1999) 
and Boff (2006). 

Within the past seventy years, the field of human safety has also gained 
prominence, as the modern consumer-driven society required more standardized 
practices and better products (Stephens & Rowan, 2006). In 1946, Amos L. 
Wood presented the first formal presentation on what is now considered system 
safety (Miller, 1966). 

Both of these well established areas of study began as grass-roots efforts 
to improve the effectiveness of the human interacting with the rest of the system. 
From the early days of human factors and system safety, additional efforts have 
arisen for better integrating human and machine, especially with respect to 
military systems acquisition. More recent efforts include the MANPRINT effort in 
the U.S. Army (U.S. Army, 1990), and what is now being termed Human Systems 
Integration—a Department of Defense (DoD) level multi-service effort to 
champion the human in systems with which they interact. Among other 
achievements, this latter effort has resulted in the establishment of a Master of 
Science in HSI available through the U.S. Naval Postgraduate School (NPS) at 
Monterey, CA. 
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C. HSI DOMAINS 

While an in-depth history of HSI can be found in Booher’s (2002) 
Handbook on Human Systems Integration, the present study requires only an 
understanding of the fields—or domains—encompassed by HSI and the inherent 
interactions between these domains. The MANPRINT program identified seven 
key domains within its purview: manpower, personnel, training, human factors, 
system safety, health hazards, and soldier survivability (U.S. Army, 2000). The 
MANPRINT handbook makes note of the existence of the interactions between 
domains; however, the extent of the information provided consists of the 
following statement and some brief examples: 

Although each of the MANPRINT domains has been introduced 
separately, in practice they are often interrelated and tend to impact 
on one another. Changes in system design to correct a deficiency 
in one MANPRINT domain nearly always impact another domain 
(MANPRINT Directorate, 2005). 

Building upon the MANPRINT effort, the Human Systems Integration 
program identified an additional area of specific importance to the U.S. Navy, but 
with reasonable application to all services—habitability (U.S. Naval Postgraduate 
School, 2006). Additionally, the NPS Master’s program focuses on the 
interactions between domains, recognizing these interactions and trade offs as a 
distinct requirement for the HSI professional. 

As the discipline of HSI is continually refined, it is possible that the number 
and definitions of HSI domains may increase, decrease, or evolve. However, 
since the aforementioned eight domains presently constitute the study of HSI, a 
brief description of each is given below. The definitions provided are taken 
directly from the MANPRINT and NPS HSI programs, and from the DAU Defense 
Acquisition Guidebook. 

1. Manpower 

Manpower addresses the number of military and civilian personnel 
required and potentially available to operate, maintain, sustain, and 
provide training for systems in accordance with Section 2434 of 
Title 10, U. S. Code. It is the number of personnel spaces (required 
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or authorized positions) and available people (operating strength). It 
considers these requirements for peacetime, conflict, and low 
intensity operations (MANPRINT Directorate, 2005). 

Manpower [comprises] the actual number of men and women, in 
the military as well as civilian and contractor personnel, required to 
operate and maintain military systems, including those personnel 
who support and provide training for the users of military systems. 
Considers the impacts of automation on both manpower utilization 
rates and on military operator-to-seat ratios in both system 
operation and maintenance (NPS, 2006). 

Manpower factors are those job tasks, operation/maintenance 
rates, associated workload, and operational conditions (e.g., risk of 
hostile fire) that are used to determine the number and mix of 
military and DoD civilian manpower and contract support necessary 
to operate, maintain, support, and provide training for the system. 
Manpower officials contribute to the Defense acquisition process by 
ensuring that the program manager pursues engineering designs 
that optimize manpower and keep human resource costs at 
affordable levels (i.e., consistent with strategic manpower plans). 
Technology approaches and solutions used to reduce manpower 
requirements and control Lifecycle costs should be identified in the 
capabilities documents early in the process. For example, material¬ 
handling equipment can be used to reduce labor-intensive material¬ 
handling operations and embedded training can be used to reduce 
the number of instructors (USD/(AT&L), 2006). 

2. Personnel 

Personnel addresses the cognitive and physical characteristics and 
capabilities required to be able to train for, operate, maintain, and 
sustain materiel and information systems (MANPRINT Directorate, 
2005). 

[The] personnel [domain consists of] the cognitive and physical 
capabilities necessary for the training, operation, maintenance, and 
support of military systems. Includes the attitudes, experiences, 
and other human characteristics necessary to achieve optimal 
system performance by matching the “right person” with the “right 
job” (NPS, 2006). 

Personnel factors are those human aptitudes (i.e., cognitive, 
physical, and sensory capabilities), knowledge, skills, abilities, and 
experience levels that are needed to properly perform job tasks. 
Personnel factors are used to develop the military occupational 
specialties (or equivalent DoD Component personnel system 
classifications) and civilian job series of system operators, 
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maintainers, trainers, and support personnel. Personnel officials 
contribute to the Defense acquisition process by ensuring that the 
program manager pursues engineering designs that minimize 
personnel requirements, and keep the human aptitudes necessary 
for operation and maintenance of the equipment at levels 
consistent with what will be available in the user population at the 
time the system is fielded (USD/(AT&L), 2006). 

3. Training 

Training is defined as the instruction, education, on-the-job, or self 
development training required providing all personnel and units with 
essential job skills, and knowledge (MANPRINT Directorate, 2005). 

Training [includes] the instruction, education, and “on-the-job 
training” necessary to provide personnel with the requisite 
knowledge, skills, and abilities needed for the correct and safe 
operation and maintenance of military systems across a wide range 
of operational conditions (NPS, 2006). 

Training is the learning process by which personnel individually or 
collectively acquire or enhance predetermined job-relevant 
knowledge, skills, and abilities by developing their cognitive, 
physical, sensory, and team dynamic abilities. The 
"training/instructional system" integrates training concepts and 
strategies and elements of logistic support to satisfy personnel 
performance levels required to operate, maintain, and support the 
systems. It includes the "tools" used to provide learning 
experiences such as computer-based interactive courseware, 
simulators, and actual equipment (including embedded training 
capabilities on actual equipment), job performance aids, and 
Interactive Electronic Technical Manuals (USD/(AT&L), 2006). 

4. Human Factors 

The goal of HFE [Human Factors Engineering] is to maximize the 
ability of an individual or crew to operate and maintain a system at 
required levels by eliminating design-induced difficulty and error. 
Human Factors engineers work with systems engineers to design 
and evaluate human-system interfaces to ensure they are 
compatible with the capabilities and limitations of the potential user 
population (MANPRINT Directorate, 2005). 

Human Factors Engineering [is] the comprehensive integration of 
human characteristics (both physical and psychological) into the 
definition, design, development, and evaluation of military systems 
in order to optimize performance in human-machine interactions. 
This includes human interaction with products, equipment, systems. 
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and environments, and addresses the capabilities and limitations or 
personnel during this interaction (NPS, 2006). 

Human factors are the end-user cognitive, physical, sensory, and 
team dynamic abilities required to perform system operational, 
maintenance, and support job tasks. Human factors engineers 
contribute to the Defense acquisition process by ensuring that the 
program manager provides for the effective utilization of personnel 
by designing systems that capitalize on and do not exceed the 
abilities (cognitive, physical, sensory, and team dynamic) of the 
user population. The human factors engineering community 
integrates the human characteristics of the user population into the 
system definition, design, development, and evaluation processes 
to optimize human-machine performance for both operation and 
maintenance of the system (USD/(AT&L), 2006). 

Human factors engineering is primarily concerned with designing 
human-machine interfaces consistent with the physical, cognitive, 
and sensory abilities of the user population (USD/(AT&L), 2006). 

5. System Safety 

System Safety is the design features and operating characteristics 
of a system that serve to minimize the potential for human or 
machine errors/failures that cause injurious accidents (MANPRINT 
Directorate, 2005). 

System Safety [is] the design of machine and system features that 
minimize the potential for human or machine errors, and of human 
and machine failures that can cause injuries. Also included is the 
ability of the system to be operated and maintained without injury to 
personnel or to other equipment (NPS, 2006). 

Safety factors consist of those system design characteristics that 
serve to minimize the potential for mishaps causing death or injury 
to operators and maintainers or threaten the survival and/or 
operation of the system. Prevalent issues include factors that 
threaten the safe operation and/or survival of the platform; walking 
and working surfaces including work at heights; pressure extremes; 
and control of hazardous energy releases such as mechanical, 
electrical, fluids under pressure, ionizing or non-ionizing radiation 
(often referred to as "lock-out/tag-out"), fire, and explosions 
(USD/(AT&L), 2006). 

6. Human Survivability 

Soldier survivability addresses the characteristics of a system that 
can reduce fratricide, detectability, and probability of being 
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attacked, as well as minimize system damage, soldier injury, and 
cognitive and physical fatigue (MANPRINT Directorate, 2005). 

Human Survivability [is] the ability of personnel to exist and function 
during and following exposure to hostile environments or situations. 
[Survivability] includes issues involving enemy and friendly combat 


weapons-induced injuries and the inherent hazards to personnel 
during threat/combat conditions, and the inherent hazards of 
military equipment (NPS, 2006). 

Personnel survivability factors consist of those system design 
features that reduce the risk of fratricide, detection, and the 
probability of being attacked; and that enable the crew to withstand 
man-made hostile environments without aborting the mission or 
suffering acute chronic illness, disability, or death (USD/(AT&L), 
2006). 

7. Health Hazards 

Health Hazards addresses the design features and operating 
characteristics of a system that create significant risks of bodily 
injury or death. Along with safety hazards, an assessment of health 
hazards is necessary to determine risk reduction or mitigation 
(MANPRINT Directorate, 2005). 

Health Hazards [includes] the identification of risk factors in military 
systems and the physical environment that can increase 
opportunities for system-caused bodily injury or death. This 
includes many inherent conditions present in the operation, use, 
and maintenance of a system (e.g., heat, cold, shock, recoil, 
motion, vibration, toxic fumes, chemical & biological agents, noise, 
radiation, etc.) that can reduce job performance and contribute to 
injury, illness, or death (NPS, 2006). 

Occupational health factors are those system design features that 
serve to minimize the risk of injury, acute or chronic illness, or 
disability; and/or reduce job performance of personnel who operate, 
maintain, or support the system. Prevalent issues include noise, 
chemical safety, atmospheric hazards (including those associated 
with confined space entry and oxygen deficiency), vibration, 
ionizing and non-ionizing radiation, and human factors issues that 
can create chronic disease and discomfort such as repetitive 
motion diseases. Many occupational health problems, particularly 
noise and chemical management, overlap with environmental 
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impacts. Human factors stresses that create risk of chronic disease 
and discomfort overlap with occupational health considerations 
(USD/(AT&L), 2006). 

8. Habitability 

Habitability [addresses] the physical living environment in which 
personnel are required to live, work, and sleep while performing 
their military duties during peace and war. This includes the 
physical and psychological needs of the individual and group, and 
takes into account morale and the social environment during both 
sustained and continuous military operations (NPS, 2006). 

Habitability consists of those characteristics of systems, facilities 
(temporary and permanent), and services necessary to satisfy 
personnel needs. Habitability factors are those living and working 
conditions that result in levels of personnel morale, safety, health, 
and comfort adequate to sustain maximum personnel effectiveness, 
support mission performance, and avoid personnel retention 
problems (USD/(AT&L), 2006). 

Habitability factors are those living and working conditions that are 
necessary to sustain the morale, safety, health, and comfort of the 
user population. They directly contribute to personnel effectiveness 
and mission accomplishment, and often preclude recruitment and 
retention problems. Examples include: lighting, space, ventilation, 
and sanitation; noise and temperature control (i.e., heating and air 
conditioning); religious, medical, and food services availability; and 
berthing, bathing, and personal hygiene (USD/(AT&L), 2006). 


D. DESCRIPTIVE ANALOGY OF HSI 

An analogy may help illustrate the interactions among the HSI domains 
and the benefit of a tool which would allow acquisition professionals to visualize 
the effects of tradeoffs between domains. Take, for example, the light bulb. 
There are a number of necessary elements which constitute an ordinary 
incandescent bulb: the glass bulb, the contents of the glass bulb, the filament, 
and an electric current. While Thomas Edison and many additional scientists 
and inventors of the 1800’s knew the individual properties of each of these 
elements, it took over 100 years to advance from Humphrey Davy’s 1809 
charcoal strip arc lamp to the incandescent bulb in use today. The significant 

advancements occurred only when the interactions between the elements were 
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taken into consideration. While Davy’s charcoal strips produced intense light, 
they burned out quickly. The enclosure of a filament in an evacuated tube 
increased its longevity. Altering the composition of the filament produced a wide 
range of bulb life. Add to this the injection of different gasses into the previously 
vacuous bulb, and the filament lasted even longer (Arizona State University, 
2006 ). 

As with most design considerations, there were tradeoffs to consider in the 
development of the light bulb. Such tradeoffs are best understood by a more 
detailed description of the latest advancements in incandescent bulbs. Filaments 
burn out as a result of evaporation, or sublimation, during use. Also, in order to 
produce the proper level of light, filaments have to be heated to a certain 
temperature. In order to increase filament life by reducing evaporation, the bulb 
was filled with inert gasses such as nitrogen and argon. However, this addition 
altered the heating characteristics of the filament. The inert gasses dissipated 
heat so efficiently that the filament could not produce a satisfactory level of light. 
An additional change had to be made in order to restore the effectiveness and 
efficiency of the light bulb. Researchers accomplished this by making a small 
alteration to the physical characteristic of the filament—winding it into a coil. This 
procedure allowed enough heat to build up around the coil to produce the 
necessary light level before the inert gasses could cool the filament. 

In a similar fashion, acquisition professionals observe the necessary 
elements of an acquisition program—one of which is inevitably the human—and 
seek the most efficient and effective combination of these elements which will 
meet the stated—and often unstated—requirements. 

It is interesting to note that modern incandescent light bulbs are not 
energy efficient. Today’s available technology is advancing the concept of using 
highly efficient light emitting diodes (LEDs) to produce equivalent levels of light 
with a fraction of the energy involved in traditional incandescent bulbs. While the 
concept is essentially the same—passing electric current through a substance to 
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produce light—modern technology is using the available resources in the most 
effective and efficient manner to date. 

E. SCOPE OF PRESENT STUDY 

Each of the domains of HSI is, of itself, a discipline with vast amounts of 
research, analytic techniques, educational programs, and methods for evaluating 
the effectiveness of the system with respect to the specific domain. Relatively 
recently, domains with a logical similarity have been the focus of interest for 
researchers studying the plausibility of creating evaluative tools which take into 
account the constraints of multiple domains. This interest has led to the creation 
of various tools with which acquisition professionals can more accurately 
determine the impact of design decisions on the system as a whole. However, 
no single tool has yet been created which takes into consideration the constraints 
and interactions of all the domains which HSI encompasses. Crisp, Hoang, 
Karangelen and Britton (2000) emphasize the importance of technologies which 
support such a total system design, especially with respect to the human 
operator. With respect to HSI evaluation, this concept may prove to be difficult, if 
not impossible, to realize. The inherent differences of each system may call for a 
different set of tools to produce an accurate evaluation. In fact, some systems 
may require the development of system-unique tools to properly evaluate the 
human-system effectiveness. However, an overarching interface which allows 
for the inclusion of any number and variety of resources utilized in the evaluation 
process would be an invaluable tool to the HSI professional. The development of 
such a tool would give decision-makers the ability to quickly and accurately 
determine the system-wide trade-offs associated with changes in a single area. 
In order for this to occur, a comprehensive study of the current tools associated 
with each of the HSI domains must be conducted. Following this study, it will be 
necessary to identify how outputs from the tools, which come in a variety of 
metrics, can be normalized in a useful manner. This step will only be realized 
after a common interface has been defined which can be configured to effectively 
convey the dynamic relationships between decision parameters. Finally, the 
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overarching interface must be properly disseminated to those conducting HSI 
evaluations. Of course, as is the goal of all efforts involving HSI, the interface 
must be intuitive, and designed with the end-user in mind. 

The purpose of the present study was to describe a process for 
conducting HSI evaluations and design the necessary resources and software to 
conduct such evaluations. The study identified the various resources currently 
available for evaluating each of the HSI domains. These resources were 
compiled in a searchable database for use by the HSI professional in the 
planning of HSI evaluations. Following a description of how HSI relates to the 
Department of Defense acquisition process, the design effort to produce an 
overarching interface was presented. This interface would allow the acquisition 
professional to evaluate the trade-offs between all relevant domains and make 
well-informed decisions with respect to the overall effectiveness of the human in 
the system. Next, a plan for insertion of the process and software into the 
acquisition community, making the tool available to all acquisition professionals, 
was discussed. Finally, as with all research, the limitations of the present study 
were discussed, as well as recommendations for future research. 

This chapter has introduced the concept of HSI, defining the common 
domains associated with HSI, and discussed the purpose of the present study. 
The following chapter discusses the effort conducted to collect information on 
HSI evaluation resources, and the result of this effort—a database for HSI 
analysts to use in planning HSI evaluations. 
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II. HSI RESOURCES 


A. DATA COMPILATION 

Compiling the information on resources available to HSI professionals is 
somewhat complicated. There are many characteristics by which the resources 
can be identified. While these characteristics may include such items as cost, 
manufacturer, and other less pertinent details, the characteristics most relevant 
to a better understanding of their usefulness to the HSI professional were 
selected as data points. The standardized format for data collection is provided 
as Figure 2. For the most part, the data collection headings are self explanatory 
and require no further description. Three headings, however, benefit from 
additional discussion. These headings are Cross-Domain Utility, Readiness, and 
Classification. 



Figure 2. HSI resource data collection format 


13 





























































1. Classification of HSI Resources 

There are myriad ways to classify the resources available to HSI analysts. 
The classification method selected needed to provide the user with the most 
relevant information based on the purpose for which the resources were 
classified. In the review of resources conducted for this study, a number of 
common terms were initially identified. These terms captured a majority of the 
available resources, and are listed below with a brief description on how each is 
differentiated from the others. 

Tool: something (as an instrument or apparatus) used in 

performing an operation or necessary in the practice of a vocation 
or profession. 

Method: a systematic procedure, technique, or mode of inquiry 
employed by or proper to a particular discipline or art. 

Model: a description or analogy used to help visualize something 
(as an atom) that cannot be directly observed. 

Database: a usually large collection of data organized especially 
for rapid search and retrieval (as by a computer). 

Questionnaire: a set of questions for obtaining statistically useful or 
personal information from individuals. (Merriam-Webster, 2006) 

The definitions above describe types of resources in a general sense. As 
one of the products of the present study would be a useful, searchable database 
of resources for HSI professionals, a more detailed classification of these 
resources would allow the HSI professional to conduct a more refined search for 
the appropriate resource. Lockett and Powers (2003) describe a classification 
process for tools pertaining specifically to the field of Human Factors 
Engineering. Tools were classified as follows: 

• guidelines and standards 

• checklists 

• subjective assessment tools 

• simulations—unmanned (with multiple subcategories) 

• simulations—human in the loop 

• miscellaneous analytical tools 
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This classification process was slightly modified and implemented in the 
compilation of the Human Factors Engineering Technical Advisory Group (HFE 
TAG) Directory of Design Support Methods (DDSM), which is reviewed in greater 
detail below (personal communication with Teresa Alley, 2006). The final 
classification structure used in the DDSM was as follows: 

• Guidelines and Standards 

• Checklists 

• Subjective Assessment Tools/Surveys 

• Perceptual Models 

• Simulation - Unmanned 

• Task Network/Workload Tools 

• Cognitive Process models 

• Graphic Human Models 

• Human Behavioral Representations (HBR) in Simulation 
Federations 

• HFE Tools Embedded in CAD/CAE Suites 

• Simulation - Human-in-the-Loop 

• Integrated Tools 

• Government Courses and Handbooks 

• Design Shells 

• Information Service Center 

• Databases 

• Other 

Since a majority of the resources reviewed in this study were present in 
the DDSM, it was determined that the same classification structure would be 
utilized in the present effort as well. 

2. Cross-Domain Utility 

One of the key tenets of HSI is an understanding of the interactions that 

exist between the identified domains. Individual resources were categorized by 

how they assist in the evaluation of the specific domain for which they have been 

designed. However, there exists the potential that a resource may provide 

additional information relevant to other domains for which the resource was not 
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initially designed. Also, the domains of HSI are not firmly set and resources may 
very well have been in existence longer than the present field of HSI. It therefore 
becomes necessary to view these resources not only with respect to their domain 
specific relevance, but more importantly, in how each resource assists the HSI 
analyst in evaluating the overall effectiveness of the system. The resulting 
information strengthens the HSI analyst’s ability to most accurately define the 
relationships between the relevant domains. 

3. Readiness 

A necessary consideration in selecting a resource to use is how quickly 
the resource can be ready for use. For example, a commercially available 
resource may be purchased, installed and ‘ready-for-use’ in a matter of hours. 
However, utilizing the resource in an effective manner may require the user to 
attend an extensive training program or to purchase additional required 
resources. With a clearer understanding of the ancillary requirements associated 
with a particular resource, the selection process would be more effective. Left 
out, the decision could result in costly overruns of both budget and schedule, as 
well as a reduction of the overall effectiveness of the HSI analysis. 


B. CURRENT REPOSITORIES 

It has already been noted that each domain of HSI is a fully developed 
discipline. As such, lists of resources for specific domains are prevalent. A 
majority of these refer to the Human Factors, Safety, Manpower, Personnel and 
Training domains, while the relatively newer areas of Habitability, Survivability, 
and Health Hazards provide few lists. Locating and including every list of HSI 
domain resources may be virtually impossible. However, many of the most 
prominent lists from around the world have been included in this study. The 
following are descriptions of each of these well-established repositories. 

1. DDSM 

The Directory of Design Support Methods (DDSM) began as an effort of 
the Designing for the User Subgroup of the Department of Defense’s Human 
Factors Engineering Technical Advisory Group (HFE TAG). The DDSM is a 
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living document, updated continuously to provide the most accurate information 
on design support tools and techniques developed by agencies such as DoD, 
NASA and the FAA, as well as those developed by NATO countries, academia 
and the private sector (DTIC-ASD, 2006). The online document is updated by 
the office of the Defense Technical Information Center-A in San Diego, CA 
(DTIC-ASD). A site map of the DTIC DDSM website is included as Figure 3. In 
addition to the current resources listed in the DDSM, those resources which are 
no longer supported, have transitioned into newer tools, or are proprietary and 
accessible only through contractual support—while not included in the report— 
are available through an online archive (Teresa Alley, personal communication, 
Sep 29, 2006). 



Figure 3. DDSM site map 

The DDSM is a great resource for the experienced FISI user, and it 
provides a wealth of readily accessible and current information on methods used 
in HSI efforts. The DDSM uses the aforementioned classification structure 
described by Lockett and Powers (2003). The standard information collected on 

each method of the DDSM includes the method title, contact information, a 
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general overview, required equipment, input/output/processing information, 
documentation including references in academic and professional journal 
articles, alternative approaches, the method’s stage of development, date of 
current version release, and validation studies. With such accurate and detailed 
information, the HSI user can more efficiently select the appropriate methods 
(DTIC-ASD, 2006). 

There are, however, limitations to the DDSM. The list is not in a database 
format and there are no search options. While originally compiled using a 
database, the database files were lost, leaving only the text format used 
currently. The user must scan the entire alphabetized list of methods or, in the 
case of a novice, search each record for the appropriate method. Fortunately, 
the current version of the DDSM is scheduled to be updated before the end of 
FY2007, with the intention of rebuilding the database (Teresa Alley, personal 
communication, Sep 29, 2006). 

2. FAA Human Factors Workbench 

The Federal Aviation Administration operates a Fluman Factors 
Workbench from their official website which boasts over three hundred human 
factors research tools and techniques. The resources are grouped into the 
following categories: Physical Ergonomics, Human Factors Knowledge, 
Knowledge Elicitation, Human-Computer Interaction, Data Analysis, Modeling 
and Simulation, Human-System Performance, Program Planning, and Safety. 
Each of these categories is further broken down in a structure unique to each 
category. The FAA Human Factors Workbench tools and techniques 
categorization is provided as Figure 4. 
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Figure 4. FAA Fluman Factors Workbench site map 


3. EuroControl 

As part of the Programme for Performance Enhancement in European Air 
Traffic Management (EATM), EUROCONTROL manages a database of tools 
and methods of human factors which are used to resolve tasks related to FISI. 
This list is available online through the EATM Ouick access link on Human 
factors Integration in Future Air traffic management systems (HIFA). HIFAdata, 
and EUROCONTROL in general, specifically target HSI activities related to air 
traffic management systems. However, as indicated on their website, the tools 
and methods can be applied to the development of “any other human-machine 
system”. The structure of the HIFAdata website is broken out similar to that of 
the FAA. General headings of Lifecycle, Tasks, Checks, Methods, Tools, 
Domains, Roles, References, and Glossary provide the HSI user with a 
convenient classification structure. Additionally, information on specific 
resources in the HIFAdata database is linked to all relevant categories. This 
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database design allows the user to identify the most appropriate resource 
through multiple avenues. A breakdown of the HIFAdata classification structure 
is provided in Figure 5. 
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Figure 5. 


EUROCONTROL HIFAdata site map 


4. HSIIAC 

The Human Systems Integration Information Analysis Center (HSIIAC) 
supports research and development efforts in HSI. This resource began in 1988 
as an effort by the Defense Logistics Agency, and has evolved over time to its 
current configuration. As of 2005, the HSIIAC is managed by the Air Force 
Research Laboratory’s Human Effectiveness Directorate, operated under 
contract to Northrop Grumman. HSIlAC’s mission is to provide analysis services 
in support of research, design and development of defense crew systems. 
Among the information on the HSIIAC is a list of available products and services. 
The majority of these are informative books or reports detailing methods for 
conducting human systems analysis. However, there are some models and tools 
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listed directly, such as the NASA TLX and a variety of anthropometric data sets. 
The website is designed to provide HSI resources by taxonomy, though at the 
present time few resources are included. The overall structure of the HSIIAC 
website is provided as Figure 6. 



Figure 6. FISIIAC FISI products and services site map 

5. Defence Research and Development Canada 

The Canadian Department of National Defence (DND) began serious 
research efforts in HSI in 1998. DND initiated an effort to identify, define, and 
coordinate the HSI process as part of the HSI-Process Models project under the 
Defence Research and Development Canada (DRD Canada). In answer to the 
original objective to “document, demonstrate, and continually enhance a set of 
HSI analysis tools and techniques as well as models, simulations and related 
databases” (Greenley, 2000), DRD Canada developed an online repository of 
HSI information. 
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Focusing mainly on the HSI process, the project limits its list of tools to 
describing those used specifically by the DND for HSI analysis, while providing 
links to additional lists and resources outside the DND. Unfortunately, the 
descriptions are somewhat limited, and only a DRD Canada contact is provided if 
further information is desired. However, beyond the limited tools listed, the site 
also contains abundant information on the HSI process as conducted by 
Canadian Forces. As much of this information is valuable to the development of 
HSI, it is included as part of the website description provided in Figure 7. The 
site is in the process of being moved, and will soon be the responsibility of the 
acquisition framework of the DND (Shaw, personal communication, Nov 9, 2006). 



6. ONR Science and Technology Manning Affordability Initiative 

The Office of Naval Research began a partnership with the acquisition 
community in 1998 to conduct research with the end goal of optimizing systems 
for the warfighter while at the same time reducing system costs and improving 
system performance. In a similar fashion to what is being done in the present 
study, one of the products of the Manning Affordability Initiative was a process 
which would “be used as a roadmap for identifying and (where required) 
developing tools and capabilities for the S&T project’s Human-Centered Design 
Environment” (DD 21/ONR, 1998). The process resulted in a collection of tools 
linked to the six-step systems engineering process as it relates to human 
engineering. The steps include Mission Analysis, Requirements Analysis, 
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Function Analysis, Function Allocation, Design, and Verification. A webpage is 
devoted to each tool and provides a general description, the related step, and 
contact information for the tool proprietor. Additionally, many of the tools include 
a link to a more detailed description. The detailed description answers a host of 
questions pertaining to the use and functionality of the tool: 

• What analyses does it conduct? What does the tool do? 

• What questions does the tool answer? 

• How complex is the tool? 

• What fidelity does the tool have to have? 

• What experience does the user have to have? 

• What are the tool inputs, and who provides them? 

• What are the tool outputs and who/what uses them? 

• What platform does the tool currently run on? What language is it 
written in? How is the data stored? 

• What other support, infrastructure, or tools are required? 

While the website is a great resource for HSI-related tools, it also contains 
excellent information on the human engineering process as it relates to systems 
engineering. Specifically, the site breaks down each of the systems engineering 
processes and details how to conduct each process with respect to the human 
element. The overall site map is provided as Figure 8. 
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Figure 8. ONR S&T Manning Affordability site map (from 

http://www.manninqaffordabilitv.com/s&tweb/index.htm) 


7. Ongoing Research 

Research is underway by the National Academy of Sciences to develop a 
study on HSI resources (John Lockett, personal communication, Sep 29, 2006). 
The Human-Systems Design Support for Changing Technology study panel was 
chartered to “develop a vision for incorporating human factors engineering 
considerations into the design process for complex systems, especially in view of 
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technologies that are changing rapidly and increasing in complexity” (The 
National Academies, 2006). One of the panel’s goals is to issue a report which 
considers “the techniques for an integrated, interdisciplinary, adaptable human- 
system design methodology and tools that can be applied in both civilian and 
military arenas” (The National Academies, 2006). The study initiated in 2005, and 
while much effort has been devoted to this endeavor, the group’s findings, to 
include databases and reports, are not expected to be available until spring of 
2007. 

8. An Explanatory Note 

It is apparent that, as with most complex systems, there is no global 
solution to the question of what resource, or set of resources, will be most 
effective. Any attempt to identify the ‘essential’ resources for the evaluation of 
HSI will inevitably limit the ability to properly evaluate a system. Each system’s 
intricate design requires careful study by properly educated and experienced 
personnel to most accurately assess the level of integration a system has 
achieved across the HSI domains. Defense Acquisition University identifies 
these individuals as HSI Analysts (USD/(AT&L), 2006). More will be discussed 
with respect to these individuals in Chapter V. Carr and Scholl (2006) describe 
the HSI process as a refining process whereby the relevant domains are placed 
under scrutiny to ensure that every possible advantage is identified and weighed. 
The products of this process are referred to as the ‘silver bullets’ of HSI. It is 
important to note that the term is used in the plural form. Each system’s unique 
design, characteristics and requirements will require an equally unique ‘silver 
bullet’. 

The database described in this chapter presents the HSI analyst with a 
starting point when planning HSI evaluations. The purpose for these evaluations, 
as well as how HSI fits into Defense Acquisition are the subject of the next 
chapter. 
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III. THE LANGUAGE OF HSI AND ACQUISITION 


A. DEFENSE ACQUISITION 

As previously noted, a key area of influence for Human Systems 
Integration is in the design, development, manufacturing and support of 
processes and products. The Department of Defense (DoD) utilizes the Defense 
Acquisition Management Framework as the means for procuring these new 
items, as well as for improving existing processes and products (Figure 9). Using 
the framework, the DoD identifies each product or process as a program, and 
assigns a program manager to oversee the procurement of the item. Program 
managers, with the support of Integrated Product Team (IPT) members, make 
the decisions affecting a program based on information about the program’s 
budget, time constraints, and key performance parameters. Since this 
information is vital in decision-making, the chief concerns of program 
management center on the cost, schedule and performance of a program 
mitigated by the associated risks. In fact, DoD acquisition professionals are 
trained on the evaluation of these three areas as they relate to a given program 
(USD/(AT&L), 2001). Often, the success of a program manager is measured by 
how well he/she manages the cost, schedule, and performance of the program. 
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Figure 9. Defense Acquisition Framework (From 
http://akss.dau.mil/dag/DoD5002/Figure1 .asp) 


The costs associated with any system are not restricted to the amount of 

money spent to buy the item. In order to better understand how much resources 
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will be required by a system over its entire life, a metric is needed which takes 
into account the cost of designing, developing and manufacturing the system, as 
well as the necessary operational and sustainment costs, and the eventual cost 
of disposal. Life Cycle Cost (LCC) is the term used to describe the summation of 
all costs associated with the system. There is sufficient anecdotal evidence that 
Program Managers have, on occasion, made program decisions based on the 
short-term acquisition cost of a system without paying attention to the associated 
future costs that would be required. In these situations, cost overruns are 
inevitable. By determining the LCC of a system. Program Managers and the 
DoD can conduct fiscal management with more accurate detail, avoiding the 
‘hidden costs’ that may otherwise surface. For clarification, the costs described 
in the remainder of this study relate to the system’s LCC. 

To assist program managers with keeping programs on schedule, within 
budgetary constraints, and meeting performance parameters, the DoD provides a 
host of courses, guidelines, best practices and additional information available 
through the Defense Acquisition University (DAU). Managing the trade-offs 
between the triple constraints of cost, schedule and performance is similar to the 
management of the trade-offs between domains of HSI. 

The DAU has implemented specific guidance with respect to the 
relationships between the three key areas of interest to the program manager. 
The DAU instruction describes the interrelatedness of cost and schedule through 
the concept of Earned Value Management (EVM) and the associated EVM 
system (Berta & Mandley, 2005). The EVM system allows program managers to 
measure the amount of work going into a project and the productivity of that work 
in dollars and hours. While the EVM system provides excellent information on 
cost and schedule, system performance, measured by the effectiveness of the 
system in meeting the key performance parameters, is tracked only in the early 
stages, or as changes to the system require. Since the human element of a 
program offers the greatest potential for improving system performance (Booher, 
2002), incorporating HSI into the acquisition process gives program managers 
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up-to-date information on the system’s performance. Acknowledging this 
opportunity, the DoD includes HSI in the latest versions of its acquisition-related 
documents. 

B. HSI IN DEFENSE ACQUISITION 

The main sources of information on acquisition in the Department of 
Defense are the DoD 5000 series documents and the Defense Acquisition 
Guidebook (DAG). Within the 5000.1 directive, HSI is implied or mentioned by 
name in the enclosure under the subsections of Safety and Total Systems 
Approach. Specifically, the document directs that program managers “shall apply 
Human Systems Integration to optimize total system performance, operational 
effectiveness, and suitability, survivability, safety, and affordability” (USD/(AT&L), 
2002a). After mentioning HSI as one of the purposes of the System 
Development and Demonstration phase in the acquisition framework, DoD 
Instruction 5000.2 includes an entire enclosure on the implementation of HSI. 
The enclosure opens with a statement that the program manager “shall have a 
comprehensive plan for HSI in place early in the acquisition process to optimize 
total system performance, minimize total ownership costs, and ensure that the 
system is built to accommodate the characteristics of the user population that will 
...[use]... the system” (USD/(AT&L), 2002b). Unfortunately, no additional 
instruction pertaining to the HSI plan is given, and the remainder of the enclosure 
describes typical activities that should be conducted in each of the individual 
domains. In effect, the instruction reduces HSI to a grouping together and 
managing of activities related to the human. The latest draft version of the 
Acquisition Strategy Guide states that “the Program Manager should pursue HSI 
initiatives within the strategy to optimize total system performance and minimize 
[total ownership cost]” (Defense Acquisition University Press, 2003). However, 
nowhere does the DoD require mandatory briefings of HSI to senior leadership at 
the major decision points of a program known as the Milestone briefings. 

However, HSI is enjoying increased visibility in the acquisition community. 

The latest version of the Defense Acquisition Guidebook (DAG) devotes an entire 
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chapter to HSI (USD/(AT&L), 2006). Building upon the information given in the 
Directive and Instruction, chapter 6 of the DAG provides a more detailed 
description of the activities associated with each domain of HSI, as well as 
detailing the need for integration of the efforts occurring in each of the domains. 
Specifically, the DAG recognizes integration as the “key to a successful HSI 
strategy”. The US Navy’s NAVAIR Acquisition Guide describes the importance 
of HSI in acquisition, with the goal “to influence system design and associated 
support requirements so that ...systems can be operated and maintained in the 
most cost-effective and safe manner consistent with manpower structure, 
personnel aptitude and skill, and training resource constraints” (NAVAIR, 2004). 

Any attempt to cohesively incorporate the efforts of HSI evaluations and 
the acquisition management strategy currently supported by the DoD must speak 
in the language of defense acquisition—that is, cost, schedule, and performance. 
Expressing HSI in these terms requires identifying the parameters to be 
measured and defining the relationship that each parameter has with program 
costs, schedule, and performance. For each program, this process is unique. 
The model described in Chapter IV provides a workspace for HSI evaluation— 
converted to the language of defense acquisition—without restricting the 
evaluator. With myriad parameters for evaluation, an example of the conversion 
of a simple parameter with respect to each domain may prove useful. The next 
section describes how each domain may be expressed in terms of one of the 
most notable dialects of defense acquisition: life-cycle costs. 


C. AN EXAMPLE: EXPRESSING THE DOMAINS OF HSI IN TERMS OF 
LIFE-CYCLE COSTS 

Cost estimating is a vital part of defense acquisition. In order to produce a 
preliminary budget, the costs involved must be estimated. Thereafter, any 
potential change to the system must be evaluated with respect to its effect on the 
program’s budget. The resulting estimates are only as accurate as the amount of 
variability accounted for by the cost estimator. Essentially, cost estimators 
attempt to define as much of the trade-space surrounding a decision as possible. 
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For this reason, a more accurate estimation of life cycle cost is achieved through 
understanding how changes affect the human element of the system. Since HSI 
in the US Navy’s SEAPRINT is currently envisioned by eight domains, the 
possible conversion of each of these domains into life-cycle costs is provided 
below as an example. 

1. Manpower 

By definition, manpower relates solely to the number of individuals 
required to operate and maintain a system (U.S. Navy, 2006). To put this in 
terms of cost we must consider all expenses associated with manpower. 
Expenses related to military manpower include wages, services such as health 
care, housing, sustenance, transportation, etc. Once these expenses are 
identified, a per-individual amount can be derived which will provide a unit cost 
for unit of manpower. In essence, knowing that a system will require twenty 
individuals to operate it, a rough estimate of the cost to supply those twenty 
individuals can be determined. 

2. Personnel 

Returning to the definitions of HSI domains, the personnel domain is 
associated with the cognitive and physical capabilities, as well as additional 
human characteristics and individual experiences necessary for a system to 
function at a given level (U.S. Navy, 2006). Costs associated with personnel 
center chiefly on the selection process and assignment of compensation. 
Determination of the level of individual attributes, or perhaps the scarcity of the 
personnel attributes required, will allow for identification of the requisite wage for 
such individuals. The cost of personnel can therefore be described in 
compensation (easily converted to dollars) per individual required. 

3. Training 

Current standard business practices call for an accounting of costs 
incurred by a company and benefits derived from those costs. The ratio of these 
two amounts, known as Return on Investment (ROI), is commonly used to 
evaluate training programs (Bartel, 2000). Total costs of training include 
development costs, lost time from performing work, overhead of education 
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department, physical materials, etc. While instructor based training may incur 
higher costs such as travel to and from training, instructor salary, and even 
refreshments, computer-based training also incurs costs stemming from media 
programming, production and distribution (Connor, 2002). 

4. Human Factors 

The field of human factors has used cost as an evaluative measure of 
usability for many years. The idea of a curvilinear relationship between dollars 
spent on usability and benefit received from such efforts is the basis for cost- 
justifying usability (Bias and Mayhew, 1994). There is a point where increased 
spending or effort results in diminishing returns. 

As with most engineering efforts, the human factors engineering industry 
estimates a certain amount of production for each hour worked. With respect to 
cost, human factors can be described as the level of effort—defined as hours 
billed—given to the human factors design of a system throughout the 
development process. The cost per hour can be calculated and, when multiplied 
by the number of hours required, will result in a cost for the human factors effort. 
In theory, the more hours dedicated to human factors design, the more usable 
the system will be. 

5. Safety 

Describing system safety as a cost measure requires an evaluation of the 
potential number of hours lost due to mishaps and converting those lost hours to 
a dollar amount by multiplying the number of hours by the per hour wage for the 
affected individual(s). This would result in the ability to ascertain the associated 
costs given the level of safety designed into a system. Increasing the level of 
safety would reduce the associated costs just as lowering the level of safety 
would increase associated costs. 

6. Health Hazards 

Within a system, each system-caused stressor adds to the level of hazard 
inherent in the operation of the system. Mitigation of these hazards reduces the 
risk of bodily injury or death. Stress fractures caused by repeated jumps with 
-I-200 pound packs, inadequate design of steam pipes in a naval vessel, and high 
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levels of vibration from sustained operations in a tank can all lead to injuries 
ranging from mild to severe and resulting in the temporary or permanent loss of 
personnel. Somewhat similar to the costs of safety described as hours lost, the 
costs of health hazards can be calculated by assessing the estimated associated 
treatment costs for level of exposure allowed in the design of a system. Added to 
this are the hours lost by such injuries, and—in the case of permanent 
disability—the costs of compensating the injured personnel and costs of 
acquiring another equally qualified individual. Exposure to nuclear, biological, 
and chemical weapons will have increased costs as the level of protection 
diminishes. As more attention is paid to addressing the health hazards of the 
system, the associated costs to the system can be reduced. 

The interrelatedness of the HSI domains is readily apparent from this 
example. Improved efforts in the area of health hazards and human factors can 
reduce costs associated with training, manpower, and safety. 

7. Human Survivability 

Survivability describes the ability of the human in the system to remain 
unharmed by non-system-related causes throughout operation of the system 
(U.S. NPS, 2006). The level of survivability afforded by the system will determine 
the number and frequency of personnel being gained by the system (through loss 
of personnel), as well as injury rate and severity from outside causes. Costs 
associated with survivability should therefore include a combination of the 
predicted probability of kill, a measurement of possible injury rate, and costs for 
acquiring additional adequate personnel and costs associated with injury 
recovery. 

8. Habitability 

Arguably the most difficult of the domains to define, habitability refers to 
accounting for the physical and psychological needs associated with sustained 
operation of the system (Naval Postgraduate School, 2006). State-of-the-art 
berthing, with soundproof sleep chambers and a schedule designed to minimize 
fatigue during the planned year-long mission, may rank low in habitability if there 
exist no means for personnel to communicate with loved ones or enjoy down- 
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time. The cost of habitability can be expressed as the costs associated with 
maintaining adequate services and facilities to induce personnel to remain as 
part of the system. 


D. THE NEXT STEP 

While the exact dimensions of the HSI foundation are still being 
deliberated, it is readily apparent that the world of defense acquisition rests 
solidly within its borders. When discussing metrics for acquisition, cost, schedule, 
performance, and risk are the essential dependent variables. Determining how 
acquisition program decisions will affect the cost, schedule or performance of the 
system allows the analyst to communicate most effectively with program 
management. 

Each defense acquisition system will require a tailored approach to HSI. 
The temptation to produce a single equation which could be applied to any 
acquisition program will not be satisfactory. However, since the concept of HSI 
is based on the evaluation of trade-offs and interactions between parameters, a 
general process and workspace within which an individual evaluation can be 
developed is critical, and will provide much needed guidance for those individuals 
conducting HSI evaluations. The next chapter presents an approach to the 
design of such a workspace. 
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IV. AN INTERACTIVE HSI VISUALIZATION TOOL FOR THE 
ACQUSITION PROFESSIONAL 

A. VISUALIZING HSI 

A logical process for the Human Systems Integration evaluation of an 
acquisition program can be summarized in four steps; planning and conducting 
the evaluation, compiling the results, presenting the findings, and receiving 
feedback from program decision-makers. Using this process as a guide, a series 
of interfaces were designed to aid both the individuals conducting the HSI 
evaluations and those who would use the results in making changes to the 
program. The interfaces were designed with the intent to be quickly and easily 
interpreted. To this end, they are highly visual in nature. The three interfaces 
include an HSI Resource Search interface, a parameter interaction editor, and 
the culminating HSI Trade Space Tool (HSI TST). Each of the three interfaces is 
described in detail below. 

B. PLANNING AND CONDUCTING THE EVALUATION 

In order for any acquisition program to be initiated, the need for which it is 
being developed must be defined. As part of this process, the Initial Capabilities 
Document (ICD) details “the need for a materiel approach...to satisfy specific 
capability gap(s)” (DAU, 2006a). When necessary, the ICD leads to production 
of the Capability Development Document (CDD). The CDD further defines the 
necessary operational mission performance parameters. Based on the human 
performance requirements identified in the program’s CDD, the HSI analyst must 
develop a plan for conducting the necessary assessments to ensure the program 
meets performance parameters. 

Similar to the HSI Resource Data Collection Sheet described in Chapter II, 
the user interface designed for planning an HSI evaluation (Figure 10) provides 
key information with respect to the myriad products available for use in 
evaluating specific human performance parameters. Whereas the data collection 
sheet allows for the input of data, the interface allows the user to search the 
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database of resources by a variety of fields: name, classification, metric, and HSI 
domain. After filling in the relevant data and executing the search function, the 
analyst is presented with a list of resources matching the query. Selecting any of 
the listed resources opens the database information on that resource, allowing 
the analyst to determine the best resource for the requirement. Once methods 
for assessing the human performance requirements have been identified, the 
HSI analyst can proceed with conducting the evaluation knowing all requirements 
are accounted for. 



Figure 10. HSI Resource Search Interface 

C. COMPILING THE RESULTS 

Once an analysis of the design decisions has been conducted on all 
relevant parameters, the HSI analyst must create a logical relationship between 
the parameters. It is at this point that true Human Systems Integration takes 
place. By identifying the relationships between parameters, the impact of design 
changes can be made much clearer. 

Considering the possible parameters for evaluation, an HSI analyst may 
be required to define a vast number of relationships in order to account for 
enough of the trade space to provide a usable solution. Looking solely at the 

36 
















two-way interactions between parameters could severely limit the effectiveness 
of the evaluation. Taking the historical conceptualization of HSI as seven to nine 
individual domains, the complete list of two-way interactions only accounts for as 
much as 17.5% or as little as 7% of the possible interactions. It would be 
irresponsible to base decisions affecting large-scale programs on such limited 
assumptions. Expanding the model to include all possible interactions (between 
128 and 502), however, would result in an excessive load for the analyst in both 
time and complexity. In order to limit the scope of the model, a fresh approach to 
the traditional domains of HSI was considered. 


Cameron and Bench (2005) propose that HSI consists of experts in the 
four domains of Manpower, Personnel, Training, and Human Factors, working 
toward goals related to the remaining domains. These four key domains exist in 
each of the services’ HSI paradigms, with the services differing in the observance 
of the final three to five domains (Figure 11). Expanding on the idea posited by 
Cameron and Bench, the four key domains can be viewed as inputs to the 
system of HSI evaluation, with the remaining domains viewed as functions of the 
interactions between key domains, and the output being total system 
performance. Focusing on these four domains as the driving force behind HSI 
effectiveness and representing the remaining traditional domains as functions— 
along with the acquisition imperatives of cost, schedule, performance and risk— 
reduces the workload to a more manageable level; defining 27 interactions and 
four functions. 


HSI Domains other than Manpower, Personnel, Training, and Human Factors 


U S. ARMY 

•Soldier Survivability 
•System Safety 
•Health Hazards 


U S. NAVY 
•Human Survivability 
•System Safety 
•Health Hazards 
•Habitability 


U S AIR FORCE 

•Human Survivability 

•System Safety 

•Occupational 
Health Hazards 

•Habitability 

•Environment 


Figure 11. 


Service specific HSI domains 
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By scoping the model to the four key HSI domains and their interactions 
with acquisition parameters, the design of the interface becomes more tailored to 
the Defense Acquisition HSI analyst. The field of HSI is still evolving, and may 
not be fully defined. Ensuring customizability of the interface would allow HSI 
professionals to experiment with optional parameters. Also, it would allow the 
interface to evolve with the current understanding of HSI. To this end, the 
interface was designed with preset parameters for each of the services as well as 
a customizable feature where the user could create and define each parameter. 

A B 



After selecting a parameter to edit from a dropdown menu in the ‘Edit 
Parameter’ box (see A in Figure 12)—or selecting to enter a new parameter—the 
HSI analyst enters information defining the parameter. This information includes 
editing the parameter name, the unit of measure, and a series of inputs 
describing the scale to be displayed. The interface was designed to allow the 
analyst to enter ranges of acceptable, questionable, and unacceptable levels 
associated with the acquisition parameters of cost, schedule, performance and 
risk. These ranges were designed to be used in the decision-maker’s interface 
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described later. The information for each parameter is saved by selecting the 
‘Save’ button at the bottom of the ‘Edit Parameter’ box. Additionally, options for 
clearing the contents or deleting a parameter are offered. 

Once this process has been completed for each parameter, the HSI 
analyst defines the interactions between parameters through the ‘Edit Interaction’ 
box (see B in Figure 12). Here, the HSI analyst can manipulate the two-way 
interactions through selecting the appropriate parameters and using one of three 
methods to describe the relationship: text input, imported data, and line 
manipulation. The text input option allows the HSI analyst to enter a 
mathematical equation which the interface then displays graphically. If the 
analyst has previously correlated the two parameters—for example, identifying 
the costs associated with certain manpower levels—the data can be imported 
directly into the interface by selecting the ‘Import Data’ button. The user is then 
prompted to identify the appropriate file which contains the XY coordinates. The 
coordinates are then displayed graphically according to the parameter settings. 
The final option for inputting relational data is to select the general type of line 
from a dropdown menu resulting in a line displayed in the graphing area. This 
line can then be manipulated by clicking and dragging points on the line. The 
interface is designed with the capability of converting the manipulated line into an 
equation which is displayed in the equation text box. Once the user is satisfied 
with the result, the interaction is saved, and the process continues for each 
interaction. 

To this point, the analyst has defined all two-way interactions between the 
main parameters and the acquisition parameters of cost, schedule, performance 
and risk. Here it becomes difficult to graphically represent the final multi¬ 
parameter interactions. The interface was designed to input these multi¬ 
parameter interactions solely through the text input option. When known, these 
multi-parameter interactions can greatly enhance the detail of the evaluation. 
Unfortunately, these interactions are difficult to define. Nevertheless, the option 
has been designed into the interface. Potentially improved methods for 

accounting for multi-parameter interactions are discussed in the final chapter. 
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D. PRESENTING EVALUATION RESULTS AND RECEIVING FEEDBACK 

The main objectives of the HSI analyst are to convey the level to which 
consideration of the human element has been designed into the system, and 
demonstrate the potential effects of system changes on the human element. 
These objectives can be accomplished through a simple graphical representation 
of the current assessment of the system, as well as the capability to manipulate 
the current levels in order to view probable outcomes. The HSI Trade Space 
Tool was designed with such an interface in mind. 

1. ‘Slider’concept 

In 2002, under a work project for the Air Force’s 311 Human Systems 
Wing, Veridian Engineering, with guidance from Air Force Major Robert Lindberg, 
developed a conceptual tool that expressed the idea of displaying, in a simple 
interface, the trade-offs among HSI domains associated with program decisions 
(Fig 13). The intent of the project was to emphasize the importance of 
understanding the bivariate relationships that exist between the domains of HSI. 
This project was a simple demonstration, and the interactions between elements 
were defined using curves derived from general observations. 


Human Systems Integration 


More at HSI CoP , 


HSIDomalns 


Welcome to the HSI relationships model. The purpose of this model is to 
allow you to better understand the relationships between each of the HSI 
domains and other key factors that are essential for a weapon system. 

Each of the HSI domains is listed on the right side of the screen. A short 
definition of the domain is displayed when you hover your cursor over it. To 
the left of each domain is a "slider" icon. Notice that they are all blue. The 
active slider will be orange (which means that none are active on the 
Welcome page!). 

Run your cursor over the orange circles below. To review the relationships 
for a specific domain area (making it active), click on its corresponding 
orange button. You should then notice that the slider icon for the active 
domain is orange. The orange slider is the "independent" domain and is the 
only slider you will be able to move. As you move it left or right, you will 
be able to "see" how it impacts the other domains ("dependent variables"). 
A description is displayed by hovering over the domain name. 

For more Information, click on "More at HSI CoP" located in the top banner. 


HSI Domains 

- Manpower 


yxxxTooooooo- 


Personnel 

Training 

Human Factors Eng. 

Environment 

Safety 

Occupational Health 
Personnel Survivability 
Habitability 


Capability 
Life Cycle Cost 


Figure 13. HSI slider tool concept (From http://Kn.qd- 

ais.com/ASPs/eLearninq/HSI Sliders/concepts/application.swf) 
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Initially, the concept of the model was based on describing the two-way 
relationships between a single domain and each of the others, as well as the 
relationships of that single domain to the life cycle cost and capability of the 
system. As discussed previously, a more accurate model was needed which 
would include multivariate interactions. While the concept could remain relatively 
intact, the processes behind it, as described in the previous section, became 
more involved. 

2. HSI Trade Space Tool (HSI TST) 

The HSI TST interface is designed using a concept similar to that of the 
slider tool developed for the Air Force. Each parameter is placed on the left, with 
descriptive labels identifying the parameter, unit of measure and present value. 
The acquisition parameters (cost, schedule, performance and risk) are placed on 
the right, with similar labels. Additionally, the text box associated with the 
present value for each of the acquisition parameters is color-coordinated with the 
present value, based on the ranges defined in the parameter editor 
(green=acceptable, yellow=questionable, red=unacceptable). The capability to 
select an individual parameter to alter is accomplished by placing a selection 
button above each parameter. Once selected, movement of the associated 
slider bar presents the predicted changes in the remaining parameters. The 
movement of the remaining parameters is designed to utilize the interactions 
defined by the HSI analyst, with the acquisition parameters defined as functions 
of the main parameters. A conceptual design of the HSI TST interface is 
provided in Figure 14. 
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Figure 14. Modified ‘Slider’—HSI Trade Space Tool (TST) 

E. JOINT DEVELOPMENT OF INTERFACE SOFTWARE 

At the same time as the interfaces and process described above were 
being developed, an effort to produce a stand-alone introductory course in HSI 
was underway at the Naval Postgraduate School by Professors Larry Shattuck 
and Nita Lewis Miller. They offered the assistance of the Office of Continual 
Learning (OCL) at NPS in developing the software for the interfaces, with the 
intent of using the process and interfaces as part of the introductory course. With 
the basic designs and detailed descriptions of the functionality of each interface 
well defined, the OCL at NPS initiated software development of the interfaces in 
late November, 2006. 

The design of the various interfaces and coursework outlined in this 
chapter would be of no use if it did not reach the audience that can most 
effectively use them. Identifying these individuals, as well as venues for 
introducing the interfaces and coursework discussed above, is the purpose of the 
next chapter. 
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V. DISTRIBUTING THE MODEL 


A. VENUES OF DISTRIBUTION 

Recent guidance to acquisition professionals identifies HSI analysts as 
those who implement HSI in systems acquisition and “assist program managers 
by focusing attention on the human part of the system and by integrating and 
inserting [domain] considerations” (USD/(AT&L), 2006). However, the current 
guidance does not elaborate on the qualifications of these individuals, how to find 
such individuals, or the process for becoming such an individual. These HSI 
analysts, as well as the acquisition community at large, are the target group for 
distribution of the HSI evaluation process and interfaces. 

Including HSI in acquisition guidance opens the door for regulation of the 
qualifications and certification of those labeled HSI analysts. Through the 
Defense Acquisition Workforce Improvement Act (DAWIA), the Under Secretary 
of Defense/Acquisition Technology and Logistics (USD/AT&L) outlines the 
education, training and experience level required for certification in many of the 
fields related to defense acquisition (i.e. Program Management, Systems 
Engineering, Contracting, Test and Evaluation). In general, certification consists 
of meeting the requirements at one of three levels. As outlined in the Defense 
Acquisition University (DAU) Catalog Appendix B (DAU, 2006), requirements for 
certification follow the general guidelines listed below: 

LEVEL I: Education — Baccalaureates Degree 

Experience —1 year experience in related field 
Training — Basic acquisition courses in related field 

LEVEL ii: Education — Baccalaureates Degree or Graduate studies 

Experience —2 years experience in the related field 
Training —Intermediate acquisition courses in related field 

LEVEL iii: Education —Masters Degree (Desired) 

Experience — 4 years experience in related field 

Training —Advanced acquisition courses in related field 
43 



A similar certification process must be included for the HSI analyst in order to 
ensure a more consistent level of expertise at the HSI analyst position. This 
process must be initiated by the establishment of a functional career field in HSI 
as has been done for all other acquisition related career fields. 

It is imperative that resources such as the ones designed in the present 
study be available to as many HSI analysts and acquisition professionals as 
possible. Widespread availability will facilitate standardization of HSI practices, 
allowing program managers to know what to expect in the way of HSI analyses. 
To this end, the key organizations and communities of such individuals were 
identified as points of entry for the software. The remainder of this chapter 
describes how the model can be introduced in the following organizations and 
communities: Defense Acquisition University, service-specific acquisition 

communities, major defense contractors, and NPS/AFIT/Service Academies. 

1. Defense Acquisition University 

The major provider of instructional material and resources related to 
acquisition is the Defense Acquisition University (DAU). The DAU provides 
coursework and instruction in accordance with DAWIA. Under DAWIA, 
acquisition professionals gain certification, and continual learning, on the best 
practices of defense acquisition. 

Incorporating the HSI evaluation process model and software into DAU’s 

coursework and website is a key step in standardization. As the hub of 

information for the DoD Acquisition, Technology, and Logistics (AT&L) workforce, 

the DAU provides a clear path to each of the services, to industry professionals, 

and to other federal agencies. Simply placing the information on the DAU 

website will not produce the desired results. Incorporating the process and 

software into required coursework will ensure that all acquisition-trained 

personnel understand—or at least have been exposed to—the HSI evaluation 

process. As mentioned in Chapter IV, coursework is currently under 

development at the Naval Postgraduate School (NPS) where the interfaces and 

process can be inserted. The products of these efforts will be a stand-alone 

introductory course on HSI as well as a four-course certificate program in HSI 
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offered by NPS. Adding these courses to the DAU catalog and including them as 
part of the basic acquisition education will disseminate the understanding of HSI 
throughout the fields of management, logistics, engineering, and analysis which 
receive training through DAU. While the coursework is presently designed solely 
as a certificate program, it may be the answer to the DAU Level I, II, and III 
training requirements for certification as an HSI analyst described previously. 

2. Service Specific Acquisition Communities 

Though all of the services rely on the DAU as a source for continuing 
education in acquisition, each service in the DoD handles the acquisition process 
differently. Regardless of the composition of the individual service’s acquisition 
force, each service educates its acquisition personnel on the service specific 
processes involved with the acquisition process. This service specific education 
includes a basic understanding of systems thinking as it relates to program 
management. As has been argued throughout this study, recognizing the human 
as a key component in the system—and the component where, through proper 
utilization, the most benefit can be gained—is vital to systems acquisition. The 
products of this study can provide the means whereby this understanding can be 
incorporated into the service specific acquisition management processes. 

3. Major Defense Contractors 

Most of the work involved with the acquisition of a system is conducted by 
defense contractors. Knowing that the more comprehensive understanding they 
have of the acquisition process makes them more appealing in the selection 
process, defense contractors attempt to stay abreast of the changing regulations 
and requirements set forth in defense acquisition guidance. In 2002, both Boeing 
and Lockheed-Martin entered into strategic partnerships with DAU in order to 
ensure best practices in acquisition management (Glass, 2002a; Glass, 2002b). 
This strategic partnership provides contractor personnel the same opportunity for 
instruction as any other defense personnel. Those defense contractors who 
have created these ties to DAU will benefit from the coursework outlined above. 
However, not all defense contractors have made these ties. Where necessary, 
providing these defense contractors with the process, software and instruction on 
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HSI will afford them the opportunity to remain competitive in the selection 
process and ensure a higher caliber of proposals where HSI is a concern. 

4. Service Academies, NPS, and AFIT 

While the greatest visibility of HSI may lie within the acquisition 
community, the concept of Human Systems Integration relates to many other 
academic degrees sought after by DoD personnel, beginning with the service 
academies. Coursework in HSI fits easily into academic departments of the U.S. 
Air Force, Naval, Coast Guard and Military Academies. Both the U.S. Military 
Academy and the U.S. Air Force Academy have Behavioral Sciences and 
Leadership departments. Additionally, each of the four service academies has a 
department or degree program related to systems engineering: Systems 
Engineering Department (USMA), Systems Engineering & Systems Engineering 
Management degree program (USAFA), Department of Weapons and Systems 
Engineering (USNA), Engineering and Management Departments (USCGA). 
Providing an understanding of HSI through introduction of the software and 
process described in Chapter IV at the undergraduate level will instill the 
importance of systems thinking—especially related to the human—that will drive 
the future of HSI. While not all academy graduates will seek higher degrees in 
HSI or Systems Engineering, they will certainly be involved in human-systems 
and/or with HSI at some point in their careers. Their previous exposure to HSI 
could tip the scales toward a human-centered decision. 

Academy graduates who desire to further explore the field of HSI should 
find the opportunity to do so within the military academic community. As 
mentioned in Chapter I, the Naval Postgraduate School offers a Masters degree 
in Human Systems Integration. However, similar to the undergraduate level 
programs, HSI can be easily integrated into other programs at NPS, such 
as the Systems Engineering department, as well as through the Air Force 
Institute of Technology’s Department of Systems and Engineering Management 
and Center for Systems Engineering. 
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5. Result of Proper Distribution 

Incorporating the process and software as outlined above creates an 
unbroken link of HSI understanding through the entire acquisition community and 
the acquisition process. The potential exists to educate all essential personnel in 
the significance of Human Systems Integration and the potential improvement 
that focusing on the human can produce. From senior leadership who determine 
new capabilities are required, to the users identified to participate in initial 
capabilities studies, to the acquisition professionals who plan for and execute the 
management of the acquisition, to the contractors who are selected to deliver the 
end produce, a clearer understanding of HSI will result in better systems. 
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VI. FUTURE RESEARCH AND SUMMARY 


A. FUTURE RESEARCH 

While the present study has described interfaces to aid HSI analysts and 
acquisition program decision-makers in communicating human effectiveness in 
the system, continued research should be conducted to ensure that this 
approach provides the most accurate description of the parameter interactions. 
Additional approaches for considering multivariate HSI trade offs have been 
suggested. These approaches include nearly orthogonal Latin hypercubes, and 
optimal computing budget allocation. 

One approach to modeling HSI involves the use of nearly orthogonal Latin 
hypercubes (Lucas & Sanchez, 2003). Through models based on this concept, 
the HSI analyst would conduct evaluations on parameters similar to the process 
presented in this study. With the results of these evaluations, and after defining 
the parameters for the model, the HSI analyst would be presented with a limited 
series of scenarios—selected through application of nearly orthogonal Latin 
hypercubes. In this manner, the HSI analyst would not need to solve every 
possible scenario. The limited series would provide sufficient statistical data from 
which any scenario could be evaluated. 

Another approach, which similarly attempts to effectively reduce the 
enormity of scenarios involved in multivariate analyses, is optimal computing 
budget allocation (OCBA) for simulation (Chen, H. C., Lin, J., Yucesan, E. & 
Chick, S. E., 2000). Through this approach, a highly efficient number of 
scenarios are determined with the intent to significantly reduce total costs 
associated with simulation. 

The present models of HSI must be refined and validated. Researchers 
must conduct detailed investigations into the interactions which exist among the 
domains of HSI—whatever those domains may end up being. Having a better 
concept of how the domains interact will reduce the uncertainty with which HSI 
analysts are forced to deal with at present. 
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In the same manner as basic use case data were collected for fields such 
as cost estimation and usability, HSI professionals must begin collecting and 
consolidating use case information on the outcomes of program decisions as 
they affect the human element. Building a database of HSI domain-assessed 
systems will provide future analysts with vital information as they attempt to 
influence program decisions in the earliest stages of system design. Similarly, 
building a database of the observed interactions between HSI system 
parameters will result in a knowledge base from which HSI analysts can draw 
when identifying human performance parameters in future system capabilities. 


B. SUMMARY 

The concept of Human Systems Integration is being included increasingly 
in the world of Department of Defense acquisition. As that role continues to be 
defined, it is imperative that processes for conducting HSI evaluations be 
standardized across the DoD. Such standardization removes the ambiguity 
surrounding HSI and allows decision-makers to know what to expect in the way 
of HSI evaluations and inputs to the acquisition process. 

The present study has defined a process whereby HSI evaluations can be 
planned, conducted, analyzed, and presented to decision-makers. Products 
currently available for conducting analyses were reviewed and included in a 
searchable database, scheduled for release via the internet on the NPS HSI 
homepage. Software for defining the interactions between relevant parameters 
of HSI, as well as an interface to display potential effects of program changes to 
decision-makers was designed and is now in development. This software will 
also be available via the NPS website when completed. Additionally, a plan for 
distribution of such tools to the acquisition community was discussed. While 
improvements to the design of specific software will always be made, the basic 
process for conducting HSI evaluations, as well as the individuals who should be 
apprised of the process and resources available, is relatively constant. 
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In an era of manpower drawdowns and continual reductions in program 
budgets, recognizing the potential for system improvement through better 
integration of the human element represents a veritable goldmine of savings. 
Omission of the burgeoning concept of HSI will result in wasted time, taxpayer 
dollars, and potentially, the very lives of service members themselves. 
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